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ABSTRACT 


On  24-27  June  1992,  the  Association  for  Computing  Machinery  (ACM)  Special  Interest  Group 
for  Ada  (SIGAda)  Artificial  Intelligence  Working  Group  (AIWG)  held  a  workshop  to  discuss 
Ada  real-time  and  AI  issues.  Panels  organized  for  the  workshop  focused  on  blackboard 
architectures,  experiences  and  lessons  learned,  real-time  development  approaches  and  issues, 
and  Ada  9X  issues  for  AI  systems.  This  paper  assesses  the  state-of-the-art  for  AI  development 
processes  and  provides  a  summary  of  the  related  workshop  discussions  from  the  experiences 
and  lessons  learned  panel.  Issues  associated  with  requirements  analysis,  design 
methodologies,  development  techniques,  test  and  validation,  and  maintainability  are  discussed 
for  AI  applications  and  for  AI  with  Ada  applications. 
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EXECUTIVE  SUMMARY 


Artificial  Intelligence  (AI)  with  Ada  is  a  reality!  Based  on  the  information  presented  at  the  1992 
Summer  Special  Interest  Group  for  Ada  (SIGAda)  Artificial  Intelligence  Working  Group's 
(AIWG)  Applications  Experiences  and  Lessons  Learned  panel,  Ada  is  a  viable  language  for  AI 
applications.  This  panel  provided  a  valuable  opportunity  to  accumulate  more  empirical 
evidence  proving  that  Ada  is  being  used  successfully  to  implement  large-scale  AI  systems. 
Congratulations  to  the  panel  participants  for  their  AI  with  Ada  successes! 

PURPOSE 

The  AIWG  formed  the  Experiences  and  Lessons  Learned  panel  to  identify  problems  and 
further  our  understanding  of  the  unique  issues  associated  with  implementing  AI  applications 
with  the  Ada  programming  language.  During  the  panel,  workshop  participants  discussed  the 
problems  encountered  and  experiences  gained  during  the  development  of  large  AI  applications 
with  Ada.  This  panel  provided  a  vehicle  for  expressing  the  AIWG's  desires  and  requirements 
with  respect  to  the  software  development  process,  changes  to  DoD-STD-2167A,  and  of  course 
Ada  9X. 

Detailed  briefings  given  during  the  panel  discussions  are  contained  in  these  proceedings. 
Highlights  of  the  panel  discussions  and  related  workshop  discussions  have  been  included  with 
the  following  message  from  the  panel  chair.  Recommendations  for  specific  AIWG  actions  are 
made  at  the  end  of  this  Executive  Summary.  A  workshop  participant  list  is  included  in 
Appendix  B. 

ISSUES 

The  panel  discussed  issues  covering  the  complete  software  life  cycle  including  the  challenges 
of  software  engineering  AI  applications;  the  difficulties  with  AI  requirements  analysis;  the 
legacy  of  an  AI  requirements  analysis;  testing,  verification,  and  validation  of  AI  applications; 
and  maintenance  for  AI  applications.  Many  of  the  issues  and  problems  discussed  during  the 
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workshop  are  applicable  to  AI  applications  developed  with  any  language  including  the  Ada 
programming  language. 

Software  Engineering 

AI  applications  typically  begin  with  few  documented  requirements  and  are  defined  as  well  as 
developed  with  a  series  of  evolutionary  prototypes.  This  is  the  current  state-of-the-art  for  AI 
applications  design  and  development  with  Ada  or  any  other  programming  language.  Software 
engineering  and  Ada  design  methodologies  typically  begin  with  a  set  of  well-defined,  testable, 
verifiable  requirements.  Therefore,  due  to  the  lack  of  requirements,  software  engineering  is  a 
challenge  for  AI  with  Ada  developers.  Without  a  set  of  well-defined  requirements,  how  do 
you  "engineer"  AI  applications?  This  question  is  the  subject  of  on-going  debates  and  research. 
However,  while  there  are  no  textbook  solutions  today,  the  information  in  these  proceedings 
describes  how  AI  with  Ada  practitioners  are  successfully  "engineering"  AI  applications  today. 

AI  Requirements  Analysis 

Requirements  for  AI  applications  are  defined  through  iteration,  that  is  learning  by  doing.  AI 
requirements  are  difficult,  if  not  impossible,  to  specify  without  prototypes.  Most  of  the  effort 
for  AI  projects  is  spent  defining  requirements  and  prototyping.  In  the  case  of  a  70,000  source 
line  of  code  Ada  application,  2/3  of  the  calendar  time  and  labor  were  expended  developing 
prototypes  to  define  the  requirements  for  the  AI  application. 

AI  requirements  analysis  requires  a  flexible  process  because  defining  AI  requirements  is  an 
evolutionary  process.  This  type  of  process  is  not  a  fixed  price  problem  and  the  current  DoD- 
STD-2167A  may  be  too  rigid  for  AI  applications  even  with  the  software  engineering  discipline 
provided  by  the  Ada  programming  language.  We  discussed  many  AI  specific  issues  associated 
with  the  software  engineering  process  and  DoD-STD-2167A.  These  discussions  led  us  to 
question  whether  AI  requirements  and  design  are  the  same  as  understood  in  the  DoD-STD- 
2167A  and  other  software  engineering  environments.  Based  on  these  discussions,  the  AIWG 
decided  to  be  more  active  in  expressing  AI  specific  requirements  and  concerns  to  the  standards 
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bodies  that  are  developing  standards  which  impact  the  development  of  AI  applications  with  the 
Ada  programming  language. 

The  legacy  of  AI  Requirements  Analysis 

Prototypes  and  human  understanding  of  the  problem  domain  are  the  standard  legacy  of  an  AI 
requirements  analysis  effort  What  happens  to  this  legacy?  We  would  like  to  use  our  human 
understanding  to  specify  the  desired  system  in  clear  unambiguous  requirements,  but  this  is 
rarely  possible.  Prototypes  typically  reflect  our  understanding  of  how  to  implement  a  solution 
and  in  many  cases  are  required  to  prove  that  it  is  possible  to  "engineer"  the  full  scale  system. 
Can  you  successfully  "re-engine “r"  a  prototype  into  a  supportable  and  maintainable  system? 
Based  on  panel  experiences  described  in  these  proceedings,  Ada  prototypes  are  being  re¬ 
engineered  into  supportable  and  maintainable  systems. 

Testing,  Verirication,  and  Validation 

The  difficulties  inherent  in  AI  requirements  analysis  inevitably  lead  to  problems  with  the 
testing,  verification,  and  validation  (V&V)  of  AI  applications.  Testing  and  validation  activities 
ensure  that  the  developed  software  system  satisfies  a  well-defined  set  of  requirements  which 
unfortunately  do  not  normally  exist  for  AI  applications.  Verification  activities  ensure  that  the 
developed  system  is  supportable  and  maintainable  which  raises  issues  associated  with  the 
feasibility  of  verifying  non-deterministic  systems  that  can  learn  and  adapt  over  time.  Testing 
and  V&V  are  critical  areas  of  current  research  because  the  public  and  the  software  engineering 
community  have  begun  to  focus  attention  on  building  trusted  systems  that  are  correct, 
dependable,  robust,  safety  critical,  efficient,  and  secure. 

The  panel  discussions  regarding  testing  and  V&V  led  to  two  interesting  thoughts.  First,  AI 
applications  seek  to  emulate  human  intelligence  and  behavior.  How  do  we  test  and  validate 
humans?  In  general,  the  proof  of  human  intelligence  and  their  ability  to  learn  is  through  on- 
the-job  performance.  Therefore,  in  essence,  humans  do  not  undergo  the  same  scrutiny  of  test 
and  V&V  that  we  are  trying  to  impose  on  AI  applications.  Second,  instead  of  trying  to  perform 
an  unnatural  test  and  V&V  scrutiny  of  AI  applications,  perhaps  it  would  be  more  meaningful  to 
develop  techniques  and  tools  that  determine  if  an  AI  application  is  fit-for-purpose;  that  is,  does 
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it  match  the  problem  domain,  is  it  operable  in  the  proposed  environments,  and  can  it  be  learned 
with  relative  ease. 


Maintenance 

Maintenance  is  an  area  of  critical  concern  as  large-scale  AI  applications  are  fielded  and  must  be 
supported  for  a  long  life  cycle.  Knowledge  base  maintenance  is  also  a  critical  concern  during 
the  iterative  development  of  large  knowledge  based  systems.  For  an  expert  system, 
maintenance  traditionally  involves  changes  to  the  rale  base  as  well  as  the  facts  which  are 
normally  considered  data.  The  discussions  in  this  area  included  questions  such  as  "What  is 
knowledge?  Are  rales  considered  data  or  software?  Should  facts  in  a  knowledge  based  system 
be  treated  as  data  or  software?". 

During  the  panel  discussions,  several  implementation  techniques  currently  being  used  for 
expen  systems  were  described  as: 

1)  implementing  the  rales  in  Ada  for  runtime  performance 

2)  use  two  modes  for  the  rale  base:  an  interpretive  mode  for  rule  execution  during 
development  and  a  runtime  mode  that  involves  translating  the  rales  to  Ada  for 
runtime  performance 

3)  a  runtime  mixture  of  interpreted  rules  and  rules  implemented  in  Ada. 

One  of  the  developers  who  is  using  the  first  approach  of  implementing  the  rales  in  Ada 
lamented  about  the  tremendous  overhead  -  approximately  1  week  -  of  recompilation  for  any 
changes  to  the  510  rales  in  his  rale  base.  The  selected  implementation  techniques  for  an  expert 
systems  rale  base  influence  both  system  maintainability  and  performance.  Based  on  their 
development  and  maintenance  experiences,  the  panel  identified  a  critical  need  for  a  support 
environment  that  includes  a  real-time  browser  for  runtime  "peeks"  into  the  system  and 
knowledge  base  maintenance  tools. 
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RECOMMENDATIONS 


Software  engineering  is  a  challenge  for  the  AI  community  due  to  the  evolutionary  nature  of  AI 
applications.  Software  engineering  is  one  of  the  strengths  and  advantages  offered  by  a 
properly  managed  Ada  environment.  We,  the  AI  with  Ada  community,  should  develop 
processes  and  tools  that  support  the  "engineering"  of  AI  applications  with  Ada.  The  AIWG 
should  add  a  section  to  the  AIWG's  annual  report  to  describe  software  engineering  processes 
for  AI  applications  and  assess  the  progress  that  has  been  made  in  the  use  of  software 
engineering  principles  to  develop  AI  applications.  The  AIWG  should  develop  a  database 
cataloging  software  engineering  processes,  tools,  and  applications  with  periodic  publication  of 
this  information  in  Ada  Letters. 

In  order  to  manage  and  engineer  Ai  with  Ada  projects,  the  AIWG  should  establish  a  set  of 
software  metrics  that  are  compatible  with  the  evolutionary  nature  of  AI  applications.  Further, 
the  AIWG  should  conduct  annual  surveys  to  determine  the  current  values  for  the  software 
metrics  so  that  this  information  can  be  used  by  the  Ada  community  to  manage  and  engineer  AI 
applications.  The  software  metrics  should  be  published  in  the  AIWG's  annual  repon  and  Ada 
Letters. 

The  AIWG  should  become  actively  involved  with  standards  bodies  that  are  developing 
standards  and  other  guidance  that  impact  the  development  of  AI  applications  with  Ada.  Of 
course,  the  first  step  in  this  process  is  to  identify  the  AI  specific  requirements  that  need  to  be 
communicated  to  the  standards  bodies.  The  issues  and  problems  discussed  at  this  workshop 
should  be  matured  into  concrete  requirements  and  recommendations  for  formal  submittal  to  the 
appropriate  standards  bodies. 

Many  of  the  issues  and  problems  faced  by  the  AI  with  Ada  community  are  the  same  problems 
faced  by  all  AI  researchers  and  developers.  The  AIWG  should  work  closely  with  the  AI 
community  to  concentrate  our  combined  efforts  on  solving  our  common  problems  rather  than 
focusing  on  the  perceived  differences  between  the  AI  and  the  Ada  communities. 
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SECTION  1 


APPLICATIONS  EXPERIENCES  AND  LESSONS  LEARNED  PANEL 
MESSAGE  FROM  THE  PANEL  CHAIR 


BACKGROUND 

During  the  past  year,  I  have  been  collecting  information  about  existing  Artificial  Intelligence 
(AI)  applications  developed  with  the  Ada  programming  language  for  the  SIGAda  Artificial 
Intelligence  Working  Group's  (AIWG)  first  annual  survey  [3].  This  was  a  challenging  task  for 
a  number  of  reasons,  but  I  have  come  to  believe  that  the  major  impediment  to  my  data 
collection  efforts  is  the  state-of-the-an  of  developing  AI  applications.  The  basic  nature  of 
specifying  and  developing  AI  systems  is  a  major  stumbling  block  to  formulating  detailed 
software  metrics  and  other  measures  that  are  so  common  for  "engineered"  systems.  AI 
systems  generally  begin  with  few  documented  requirements  and  ?re  developed  with  a  series  of 
evolutionary  prototypes.  Typical  software  metrics  are  not  readily  available  for  AI  systems. 

This  fact  led  me  to  investigate  the  current  processes  used  to  develop  AI  applications  and  the 
problems  of  applying  current  software  engineering  principles  to  AI  applications. 

This  briefing  was  presented  at  the  general  session  of  the  Summer  '92  SIGAda  conference.  I 
presented  this  material  as  a  devil's  advocate  challenge  that  stated  the  issues  in  an  effort  to 
generate  open  discussions  about  the  issues.  These  and  many  other  issues  were  discussed 
during  the  workshop.  For  the  publication  of  this  briefing  in  the  workshop  proceedings,  I  have 
added  relevant  information  from  the  workshop  discussions  to  the  briefing.  My  briefing 
combined  with  my  interpretation  of  the  workshop  discussions  is  the  method  I  have  chosen  to 
document  the  workshop  discussions  for  these  proceedings.  Appendix  A  contains  my  briefing 
and  the  workshop  discussions. 
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APPENDIX  A 


SIGAda  Artificial  Intelligence  Working  Group 
Summer  '92  SIGAda  Workshop 

This  paper  assesses  the  current  state-of-the-art  for  Artificial  Intelligence  (AI)  applications 
development  processes.  This  paper  discusses  the  human  and  safety  reasons  why  we  need  to 
"engineer"  AI  applications  and  presents  information  about  the  current  processes  used  to 
develop  knowledge  based  systems.  Issues  associated  with  requirements  analysis,  design 
methodologies,  development  techniques,  test  and  validation,  and  maintainability  are  discussed 
for  AI  applications  and  for  AI  with  Ada  applications. 

This  paper  uses  viewgraphs  with  an  accompanying  discussion  section  that  is  divided  into  two 
parts.  The  first  states  current  issues  while  the  second  part  provides  a  summary  of  the  related 
workshop  discussions. 
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software  engineering  principies  when  we 
write  software  to  solve  our  problems. 
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Nancy  Leveson,  May  1992.High  Pressure  Steam  Engines  and 
Computer  Software  Keynote  address  at  the  14th  International 
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Software  Safety  and  Liability  Issues 

Our  ability  to  "engineer"  AI  applications  is  rapidly  becoming  a  deciding  factor  in  the  acceptance  of  safety-critical  systems 
that  contain  AI  applications.  After  several  decades  of  successful  research  and  prototyping,  we  are  just  beginning  to 
grapple  with  the  issues  of  building  "trusted  systems"  that  are  safety  critical,  correct,  dependable,  robust,  efficient,  and 
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Nancy  Leveson,  May  1992.Hiah  Pressure  Steam  Engines  and  Computer  Software 
Keynote  address  at  the  14th  International  Conference  on  Software  Engineering, 
p  2  -  14  of  the  Proceedings. 
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Ada  provides  an  optimal  software  engineering 
process  for  all  Artificial  Intelligence  problems. 


Theorem:  An  optimal  software  engineering  process  exists  for  all  Artificial  Intelligence  problems 

Our  greatest  challenge  is  the  application  of  software  engineering  principles  to  the  development  of  AI  applications.  This 
theorem  and  proof  are  my  way  of  asking  the  question: 
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This  "design  for  evolution"  is  accomplished  by  conforming  to 
standards  (UNIX,  FDDI,  Ada,  X  Windows)  and  designing  the 
interconnections  with  sufficient  bandwidth  to  allow  for 
faster  components  as  they  become  qualified  for  space. 

■  ■  • 

Donald  Woods,  April  1992,  Al  Expert,  Vol  7,  No  4,  pp  32-39, 


Our  Challenge:  Software  Engineering  with  Ada  and  AI 

The  future  is  bright  for  both  AI  and  Ada.  What  does  the  future  hold  for  AI  with  Ada?  The  answer  to  this  question  will 
depend  on  our  ability  to  use  sound  software  engineering  processes  in  the  development  of  AI  systems  with  Ada  --  or  any 
other  language. 
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Knowledge  Based  Systems  and  Ada:  An  Overview  of  the  Issues. 
Ada  Letters,  p  72  -  81. 


Can  we  bridge  the  gap  between  the  AI  and  Ada  cultures? 

The  representation  of  knowledge,  manipulation  of  knowledge,  reasoning  with  knowledge,  understanding  the  true 
meaning  of  intelligence,  and  the  impact  of  our  systems  on  intelligent  beings  are  some  of  the  exciting  challenges  facing 
all  Artificial  Intelligence  (AI)  researchers  today.  These  challenges  form  a  common  bond  between  the  AI  with  Ada 
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Phillippe  Collard  and  Andre  Goforth,  November/December  1988, 
Knowledge  Based  Systems  and  Ada:  An  Overview  of  the  Issues. 
Ada  Letters,  p  72  -  81. 


Requirements  Deflnition  with  Prototypes 
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Are  AI  with  Ada  systems  testable,  maintainable,  and  reliable? 

The  difficulties  inherent  in  AI  requirements  analysis  inevitably  lead  to  problems  with  the  testing,  verification,  and 
validation  (V&V)  of  AI  applications.  Testing  and  validation  activities  ensure  that  the  developed  software  system  satisfies 
a  well-defined  set  of  requirements  which  unfortunately  do  not  normally  exist  for  AI  applications.  Verification  activities 
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Case  Study:  Accuracy  of  Existing  Systems 

Culbert,  Hamilton,  and  Kelly  documented  the  experiences  and  problems  encountered  by  knowledge  based  systems 
developers  and  users.  In  their  paper,  the  terms  knowledge  based  systems  and  expert  systems  were  used  interchangeably 
by  the  authors.  Although  the  focus  of  the  authors  was  verification  and  validation,  the  article  reveals  some  very  valuable 
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Case  Study:  Design  Methodologies 

Many  of  the  survey  respondents  did  not  use  a  formal  design  and  development  methodology.  What  are  the  design 
methodologies  used  to  develop  AI  with  Ada  applications? 
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As  discussed  in  the  survey,  70%  of  the  systems  were  operational  and  the  rest  were  considered  prototypes  but  some  of 
these  prototypes  had  operational  users.  The  respondents  could  not  clearly  distinguish  between  prototypes  and  operational 
systems  --  their  systems  were  operational  prototypes.  How  does  this  "reality"  fit  with  the  Ada  design  and  development 
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